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ABSTRACT 

Mita the costs of pilot training escalating, it is 
becoming increasingly important to make as few mistakes as 
possible in the selection of potential aviators. In the 
early days of aviation the use of psychomotor testing played 
a big role in this selection process, but the physical 
complexities of the system caused its discontinuance. More 
recently, researchers at the Naval Aerospace Medical 
Research Laboratory, using micro-computers, have developed 
two new series of psychomotor tests. This thesis uses 
stepwise and multiple regression techniques to confirm the 
Viability of using such a series of psychomotor tests to 
predict the flight grades of student aviators in primary 
flight school. The fitted regression model accounted for 
77% of the variance in the primary flight grade data 
examined and appeared to be approximately 4.5 times better 


than the model currently used. 
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IRE INTRODUCTION 


A: ENLKODUC TION 

Since the middle of World War I there has been a 
concerted effort to predict the physical skills and mental 
facilities necessary to ensure completion of the rigorous, 
and often dangerous, pilot training syllabus. Batteries of 
tests, mostly of the "paper-and-pencil" type (such as the 
Navy's Flight Aptitude Rating(FAR) and the Academic 
Qualifying Test(AQT) which are currently used), have been 
developed and used exclusively since World War II, with 
limited success. ENEE costs of pilot training 
escalating to phenomenal levels ($804,793 for the jet 
aircraft training pipeline) it has become increasingly more 
important to improve the predictive validity of the aircrew 
selection process. [Ref. 1] 

To this end, the Naval Aerospace Medical Research Lab 
located at the Naval Air Station in Pensacola, Florida has 
developed and administered a number of computer-assisted 
divided-attention and undivided-attention psychomotor tests 


to volunteer Student Naval Aviators (SNA) and Student Naval 


ENSht Officers (SNFO), prior to their entry into formal 
Wight training. The results of these tests have been 


compiled with individual results from primary flight school, 


SNA/SNFO demographic information (degree major, age, sex, 
etc.), and FAR/AQT scores. 

The purpose of this thesis is twofold: 

1) to produce a model which will indicate the value of 
reinstituting psychomotor testing US GMC aoe 
computer technology; and, 

2) to analyze the validity of psychomotor testing as a 
predictor of flight grades during the primary flight 
training syllabus. 

B. BACKGROUND 

Just two years prior to the United States' entry into 
World War I, the Aviation Service had but 52 fliers on its 
roster. By April of 1919 the number had grown dramatically 
to approximately 19,000. [Ref. 2:p. 103). With this growth 
was a growing concern for the methodology in selecting the 
"right person" for the demanding, stressful, and often 
hazardous training. 

Early procedures, more esoteric than scient ZE 
required only that the future aviator be "of good education 
and high character, men who were in every way qualified and 
fitted to become officers of the U. S. Army." [Rehm 
103] These selection criteria, which were the same for the 
cavalry, were completely arbitrary and non-quantitative, 
leading to inconsistency in aviator selection. The high 
accident rate during flight school made it apparent that 
these gualitative procedures were adeguate for "ground 


positions", but did nothing to predict the behavior meen 


abilities of these men once they entered the cockpit. The 
need for a well organized predictive series of examinations 
began to emerge. 

The first series of examinations that were developed 
were utilized until the end of the 1940's, and were 
moderately successful, involving the use of tests for mental 
stability and psychomotor skills. The psychomotor tests, 
however, were awkward and difficult to carry to each entry 
station. The Mashburn Automatic Serial Action Apparatus, as 


depicted in Figure 1, consisted of a frame 60 inches long, 


29 inches wide, and 18 inches high, in which were mounted an 


meagustable seat and airplane controls (i.e., stick and 
rudder). Insftront orsthezeentrols Was an Upright section 64 
inches high and 36 inches wide [Ref. 3]. In addition “this 


equipment required special training for assembly and for 
accurate collection of performance data. By 1951, the many 
administrative, reliability, and quality control problems 
MN this tps? testing to be discontinued by all 
services [Ref. 4]. 

Attempts to reinstate psychomotor testing have failed 
over the past nearly 40 years. Written tests designed to 
"imitate" psychomotor tests proved inadequate, measuring 
only the most basic of dexterity and coordination skills. 


In its place appeared the "perceptual/cognitive tests", such 


as the Spatial Apperception Test, which have changed little 


in spite of the vast advances in technology. 










AILERON RESPONSE AILERON STIMULUS LIGHTS 


LIGHTS 


ELEVATOR RESPONSE ELEVATOR STIMULUS LIGHTS 


LIGHTS 


RUDDER RESPONSE RUDDER STIMULUS LIGHTS 





LIGHTS 
CONTROL 
eO P. 
RUDDER 
Figure 1. Mashburn Automatic Serial Action Apparatus 


er: PRESENT DAY TESTING 

Current testing of potential aviators begins and ends at 
the Naval Recruiting District, where the battery of officer 
selection tests is administered. This sequence of tests 
consists of three timed and one untimed examinations. 

The timed tests last no longer than 110 minutes and test 
for general intelligence, general mechanical aptitude, and 
morlrty to orient in space. The remaining "test" is 
untimed and aimed at the evaluation of specific elements of 
personal history, interest, and general knowledge of 
aerospace. These tests (the Academic Qualification Test, the 
Mechanical Comprehension Test, the Spatial Apperception 
Test, and the Background Inventory) are described in more 
detail in Chapter Two of this paper. Sample questions for 
Each of these tests are found in Appendix D. 

Once the tests have been completed or the time limit for 
each has expired, they are hand graded, twice, by recruiting 
personnel. Within five days the tests are forwarded to Naval 
Aerospace Medical Institute (NAMI) in Pensacola, Florida for 
munmBher validation. 

Test scores are combined into two ratings which are used 
as two of the criteria for the selection of potential 
officer candidates. The raw scores of the Academic 
Qualification Test and the Mechanical Test are added to 


yield the Officer Aptitude rating (OAR). Using Table I this 


score is converted into the final OAR. The Flight Aptitude 
Rating (FAR) is calculated by adding the converted scores of 
the Mechanical Comprehension Test, the Spatial Apperception 
Test, and the Biographical Inventory (Tables II, III, and 
IV) This score is further converted using Table V 
generating the final FAR score. 

Minimum acceptable scores for acceptance into flight 


training are: (read: Pipeline/GPA/AQT/FAR/OAR) 


Pilot/279/3/5/40 


Other Aviation {NFO,ete:) 72707372728 


The ease associated with the administration of written 
tests is offset by inherent drawbacks. Since these tests 
are "graded" at the Recruiting District, the test keys are 
susceptible to compromise. Additionally, study books, such 
as the ARCO "Officer Candidate Tests" Study Book, may 
prepare potential aviators for the examination, but may not 
improve their skills in the air. 

Computer-assisted psychomotor testing, on the other 
hand, tests the potential aviator's innate ability to handle 
minute changes in direction, speed, and altitude. Should 
computer programs which emulate Navy testing be sold 
commercially to "tutor" prospective aviators, the result 
could produce potentially better fliers, a most desirable 


effect. 


DORPER OrriCheRe APTITUDE RATING SCORE CONVERSION 


Raw Converted Raw Converted 
164/above 80 116=117 49 
163 78 115 48 
EZ 76 quisa 47 
161 74 112 46 
159-60 703 1210111 45 
T58 72 108-109 44 
257 70 205-107 43 
156 69 104 42 
155 67 1022103 41 
253-154 66 99-101 40 
517-152 65 97-98 39 
2483-150 64 96 38 
247 63 93-95 37 
144-146 62 91-92 36 
811—143 61 89-90 25 
139-140 60 87-88 34 
138 59 DOLOS 33 
1936-137 58 Ba un 32 
34-135 57 82 AJ 
ml 133 56 2083 30 
829-130 55 nuc 28 
E26—127 54 75-76 27 
124-125 53 74 26 
E2- T23 52 73 24 
220-121 52 T2 2 
118-119 50 71/below 20 


MABLE II. MECHANICAL COMPREHENSION TEST SCORE CONVERSION 


Raw Converted Raw Converted 
70/above 19 43-45 9 
67-69 18 40-42 8 
65-66 o) 38-39 ji 
63-64 16 25537 6 
61-62 qo 85314 5 
59-60 14 UE 4 
57-58 13 27-30 3 
656 12 DA-26 2 
59-52 T1 23/Below 1 


46-48 


TABLE II 


TABLE IV: 


Raw 


59/above 
55-58 
52-54 
50-51 
46-49 
43-45 
40-42 
37-39 
34-36 
31.33 


TABLE V. 


SPATIAL APPERCEPTION TEST SCORE CONVERSE 


Converted 


Raw 


l6— 
14-15 
125998 
TO HN 
Sag 
9 
5-6 
a 


3/Below 


Converted 


RN 60 ds 01 ON SI 00 10 


BIOGRAPHICAL INVENTORY SCORE CONVERSION 


Converted 


Raw 


25-30 
26-27 
24-25 
al 22 
19520 
ise 
15-16 
13-14 


12/Below 


Converted 


PD tM O 0 MJ 0 10 


FLIGHT APTITUDE RATING SCORE CONVER TON 


Raw 


42/above 


29=41 
36.38 
22235 
29231 
25-28 
22-24 
T9221 


18/Below 


F2 N C5 Ud -J CO AO 


Converted 


E. TESTS 


The data used in this study were taken from the series 
of pen-and-paper and psychomotor tests discussed below. 
These data were later analyzed for inclusion in a regression 
model used to exam the validity of psychomotor testing as a 
tool in predicting the flight grades achieved by student 
Bx tors during primary flight training. The exact 
Session model is discussed in greater detail in Chapter 


Mot this study. 


foe THE SUBJECTS 

All subjects were volunteers awaiting assignment to one 
of three flight training squadrons (VT-2, VT-3, or VT-6) at 
Being Field, Milton, Florida. Of the 111 original 
participants, 55 were chosen at random as the basis of this 
study. The remaining 56 were used for cross-validation of 
the model. The selection was performed using the random 
number generator in STSC, Incorporated's APL*PLUS APL 
language package. A random seed of 1,153,851,501 was used. 
Of the 55 chosen Student Naval Aviators, 52 were male and 
three were female, with ages ranging from 21 to 28 years 
(Median = 23 years, Mean = 23.27, SD = 1.15). The majority, 


47 , nad no previous logged flight time (Minimum = 0 hours, 


Maximum = 90 hours, Median = 0 hours, Mean = 6.05 hours, SD 
= 197 517 hourse 

All subjects had obtained at least a baccalaureate 
degree, with the majority having a degree in mathematics- 
specific programs, as indicated by Figure 2. All were 
commissioned officers, the majority of whom received their 
commission through Aviation Officer Candidate School (Figure 
3l: 

Each of the participants had completed either Aviation 
Officer Candidate School (AOCS) or Aviation Indoctrinau 
classes. These schools offer only technical aviation 
instruction, such as Air Navigation Theory MEES 
Aerodynamics, and Aircraft Engineering, and do not involve 


actual “in-flight” aviation training. 


Engineering/Math XXXXXXXXXXXXXXXXZXXx x ee 


General Science XXXXXXXX (8) 
(Biology, ete.) 


Business XEXXXXXXXXXxXxX (13) 
Social Science KXXXXXXXXXXx (11) 


Physical Education ls) 


Figure 2. Degree Majors Of Participants; 
(Parenthesis indicate number of Participants with Degree in 
that field) 


10 


After the experimental tests, all volunteers went to 
Die nem Le MS: primary flight training. The distribution 
of the subjects across training squadrons is represented in 


Figure 4. 


Aviation Officer 
Candidate School XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX (31) 


United States 
Naval Academy XXXXXXXXXX (10) 


Reserve Officer 
raining Corps XXXXXXXXXXX (11) 


Other (Direct 
commission, etc.) aled) 


Figure 3. Source Of Commission 
(Parenthesis indicate number of Participants with 
Degree in that field) 


UU XXXXXXXXXXXXXXXXXXXXXXXXXXX (27) 
VT-3 XXXXXXXXXXXXXX (14) 


VT-6  XXXXXXXXXXXXXX (14) 


Eucure 4. Distribution Of Subjects Across Training 
Squadrons 
(Parenthesis indicate number of Participants with Degree in 
that field) 
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BRSEARPPARATUSFZNDETESTS 
The tests and their associated apparatuses can be 
divided into three distinct catecoqommece Pen-and-Paper 
Tests, Gibb-Damos Computer-Based Tests, and the Griffin- 
Mosko Computer-Based Tests. 
1. Pen-and Paper Tests 
These tests are currently used for the selection of 
Aviation Officer Candidates and Naval Flight Offa. 
Candidates. They are, generally, administered at the Naval 
Recruiting Center located in the region of recruitment. 
a. Academic Qualifying Test (AQT) 

The AQT is a test of general intelligence, 
measuring verbal aptitude (i.e.,Vocabulary) and quantitative 
aptitude (i.e. arithmetic reasonin DE Scores are discrete 
measures, ranging from a low of 1 to a high of 9. This test 
is administered at the Naval Recruiting Center. 

b. Mechanical Comprehension Test (MCT) 

Administered at the Naval Recruiting Center, 
this test measures the subject's ability to understand 
physical relationships and the principles involved in the 
operation of mechanical devices (i.e., basic physics). 
Scores range from a low of 1 to a high of 19. 

c. Biographical Inventory (BI) 
This test is presented in questionnaire format 


and has no” timer limas Emphasis is placed on: basic 


T2 


knowledge of aerospace history, technology, and terminology; 
Bey sonarsopınlons; (arcasgof interest and attitudes. No 
emgle item is heavily scored or significant in itself. 
Administered with the MGT, scores range from a low of 1 to a 
bugh of 19. 
d. Spatial Apperception Test (SAT) 
Administered along with the MCT and BI, this 
“Ss the candidate's ability to orient in space or, 
specifically, to visualize the spatial relationship between 
me attitude of a plane and the territory over which it 
flies. Scores range from a low of 1 to a high of 19. 
2. Gibb-Damos Computer-Based Tests 
a. Apparatus 
This battery of tests was conducted on a Apple 
TLE micro-computer connected to Amdek Color I Plus monitor 
and Apple IIe numeric keyboard. The control stick, used to 
emulate an aircraft control stick, was a Measurement Systems 
Incorporated 542 cursor control unit mounted on a chair and 
located between the legs of the subject. 
b. Tasks 
There are two basic tasks involved in this 
series of tests: One-dimensional Compensatory tracking and 
Absolute Difference. These two tasks are performed 
independently and then concurrently in a dual-task format. 


For the Absolute Difference tests, the number of correct and 


T3 


incorrect responses, as well as the associated response 
times, was recorded. There were no timed responses for the 
compensatory tracking tasks. 

(1) Absolute Difference Tasks. Randomly selec- 
ted digits ranging in value from 1 to 9 were displayed at 
the center top Orf thesmonıterm The subject mentally 
calculated the absolute difference between the digit 
displayed and the immediately preceding digit and entered 
this value on the numeric keypad. The only valid responses 
were the integers 1 through 4. Each session had a duration 
of 2 minutes with a 15-second rest between sessions. Each 
subject completed 10 such sessions. 

(2) One-dimensional Compensatory Tracking. This 
task description is quoted from Gibb-Damos [kef. 6:2 «XT 

The subject was required to keep a 0.6-centimeter square 
centered in a 9.75-centimeter by 1.25-centimeter rectangle 


by making appropriate left-right movements of a control 
stick. The cursor was driven by a forcing functi 


consisting of equal amplitude broadband noise.... This 
task was controlled by the subject's left hand. The 
subject received five 2-min trials. Each trial was 
separated by a 30-s rest. The dependent measure was RMS 
error, With the control stick displaced as far mois 
possible to one side throughout the trial, the average RMS 
errori Was 0250 With no control inputs, the average RMS 


error score was 78. 


(3) Dual-task Tracking-Absolute Difference. 


During the dual task trials, the absolute difference numbers 
were displayed at the center-top above the tracking task. 


The subject was informed to place equal emphasis on each of 


14 


the the tasks. The same measures as the single-tasks 
versions were used. There were a total of five 2-minute 
sessions with approximately 30 seconds between each trial. 
cele; t tinsMesKONCompütereBasedi Tests 
a. Apparatus 

Test eguipment for this series of tests was 
similar to that of the Gibb-Damos tests. The major 
differences were in the additional axes for the rudder 
simulation and throttle simulation. The Dichotic Listening 
Tests were performed with the aid of a dual-channel tape 
recorder and binaural headphones (Figure 5). 

b. Tasks 

For this battery of tests there are, again, only 
myo basic parts: The Dichotic Listening Tests and the 
Multidimensional compensatory tracking tasks. As in the 
Gibb-Damos series, these two tasks were performed singly and 
then as a dual-task. At the beginning of this battery of 
tests the subject was presented with an example of each of 
the tasks, and was afforded the opportunity to contact the 
test administrator should any question or problem arise 
prior to the start of the tests. 

This task, representing the communications and 
attention management component of the simulated flight task, 
consisted of two separate sets of letter-digit strings read 


Simultaneously over the binaural headsets, one string heard 


15 


in the left ear and one string heard in the richt ear REN 
subject was presented with 24 trials during the single task 
session and was instructed to respond only to the digits 
heard in the designated ear, while ignoring the letters and 
digits heard in the "non-designated" ear. A sample of the 
DLT letter-digit string is presented in Figure 6 [Ref o 
37% The number of correct responses was recorded as the 
measure of performance. 

(1) Multidimensional Compensatory Tracking se 
this tracking task, the subject views a video screen 
displaying a "cross-hair" pattern. A cursor was forced 
vertically and horizontally from the center ™ of the 22658 
hair pattern. The subject was required to keep the cursor in 
the center position using a control "joy-stick mene 
movements are opposite of the anticipatory movement of the 
GWESOEL For instance, when the joy-stick was moved to the 


right, the cursor moved to the Jere. 


LG 


YH 


eee PT ATCT oe 





Gua Dr UE Ein MOS KO Experimental Apparatus 


dei 


Part I 

(Left Ear) R8 NSMY2G B7 E E'N 
"RIGHT" (Vocal Channel "attend" Command) 

Right Ear) YL 3 SR 4FZ 9 NE NE ——— 


i. ft Ear) BF 4 3 7 9 
"LEFT" (Vocal Channel "attend" Command) 
(Right Ear) G 1752652 


Correct Responses 
AO ATS TT 


Figure 6. DLT Example 


(2) Dichotic Listening Task (DLT). A second 


cursor, simulating rudder movement, was positioned at the 
bottom of the screen and moved only along the horizontal 
axis. It was controlled by rudder-like pedals which, again, 
moved the cursor in the opposite direction from ches 
anticipated. The last cursor moved vertically along the y- 
axis and was controlled by a joy-stick whose movements are 
normal with respect to the cursor (move the joy-stick 
forward, the cursor goes down.) The first test involved 
only the horizontal/vertical (X and Y) axes, and simulated 
aircraft control stick movement, solely. There were a total 
of two sessions lasting 3 minutes each. The second single 
task test was a simulation of aircraft stick and rudder (Xj 
Y, and Z axes), with three sessions lasting 3 minutes each. 


The last single task simulated aircraft stick, rudder, and 
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throttle movement and control (X, Y, Z, and T axes). There 
were two 3-minute task sessions. The error score was 
derived automatically from .01 inch deviations from the 
"target" position on all axes. Average error for each 
minute of each session was recorded. Cumulative error for 
each axis during the session was recorded at the termination 
of each session. 

(3) Dual-task Tracking-DLT. Subjects performed 
the DLT simultaneously with all the tracking combinations 
mentioned above, except the stick, rudder, and throttle (X, 
Bez, and T axes). The number of DLT trials was held to 12, 
as compared to the original 24 for the multi-task sessions. 
Me presentation of the DLT letter-digit strings began 30 
seconds after the beginning of the tracking tasks, and ended 
1.5 seconds before the end of the tracking tasks. The 


performance measures for both remained the same. 
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III. DISCUSSION OF THE DATA AND EXPLORA eee eee E 


The initial exploratory analysis of the data was done 
using the capabilities of the APL-PLUS and STATGRAPHICS 
personal computer software packages. After the preparation 
of the data, which is described in detail in this chapter, 
each variable was evaluated against the dependent variable 
"Primary Flight Grades." Unless otherwise noted all results 


are quoted at the .05 significance level. 


A. DATA PREPARATION 

The data was provided in a matrix consisting 0666655 
observations each containing 172 variables. Most SETE 
variables were multiple observations of tracking data with 
the X-axis, Y-axis, and Z-axis each represented by a 
different variable. 

The first transformation of the data was performed on 
the tracking data. Using the simple Pythagorean equation, 
presented below, the X, Y, and Z coordinates were translated 


Into a Euclidean distance: 


Tracking Error = ,/X2 € Y2 0: 


where: X the X axis coordimae 


In the Y axis coordinate 


Z 


the Z axis coordinate (when available) 


ae 


Wééé-.Wéda-tance summarızes the tracking error for that task 
from the origin, measured in computer Graphics “pixels”. 
After the distance was computed, each of the trials was 
averaged to yield a single performance summary for that 
specific tracking task. 

Other multiple observations, such as the Dual Task 
Dichotic Listening Test and the Single Digit Absolute 
Difference Test, were averaged to obtain a single 
representative number or summary. 

The transformations and groupings resulted in the 
reduction of the number of variables from 172 to 41. The 
Anal list of variables and the original list are both 
presented in the appendix. Note that there are 42 variables 
listed vice 41 variables, the first of which is the subject 
number. All variable numbers listed throughout the analysis 
portions of this study may be cross-referenced with the list 
in Appendix B. 

[Miiemlast Variable, VAR42, was the tracking error 
Mmegtlizing the throttle dimension (X/Y/Z/T axis tracking.) 
Since there were not enough observations with this variable 
to produce a model and perform cross validation it was 
deleted from this study. This in no way reflects negatively 
on this variable, which may prove to be a valuable input to 


later models. 
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Be EZPLORATORTERNZTYSTS 
During this phase of the analysis it was thought prudent 
to see if there was an "obvious" relationship between any of 
the myriad variables available and the primary flight grades 
of the subjects. The initial analysis follows: 
1. Primary Flight Grades vs. AOT(VAR8) and FAR(VAR9) 
scores 

As seen in Figure 7 there appears to be little 
predictive nature to the AQT scores. At first there appears 
to be a small curvature (e.g. quadratic) effect, but this is 
not borne out in later analysis. Since those subjects whose 
AQT scores were at level 4 performed approximately as well 
as those whose score was the highest value of 9, this 
variable was not considered to have a significant effect on 
the flight scores. 

In case of the FAR scores depicted in Figure 8, 
however, there appears to be a slight trend as the score 
increases. Those with scores in the 7-9 region appeared to 
score higher in flight school 

2. Primary Flight Scores vs. Trackin nerio 

In all cases the tracking error scatter plots were 
not helpful in establishing a perceivable trend. Figure 9 
provides a representative plot. In this plot the Prime 
Flight Scores are plotted against the one dimensional 


compensatory tracking error (VAR21). Notice that there 
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Figure 9. Scatter Plot Of Primary Flight Grades 
vs. Tracking Error 


appears to be no discernible pattern or trend to the data. 
The remainder of the tracking error scatter plots ebe 
similar behavior. These plots can be found in Appenci SEE 
3. Primary Flight Grades vs. Remaining Variables 

These ‘plots, like those of the tracking CE 
displayed little, if any, pattern indicating sige 
relationship between the variable and primary flight grades” 
In Figure 10 the number of incorrect responses oota 
during the single task absolute difference (VAR25) teste 


plotted against flight grades. 
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Eerst Plot Of Primary Flight Grades 
vs. Number Wrong On Absolute Difference Test (VAR25) 


e ES Ub On Or Primary Flight Grades 


During the initial analysis of the flight grades, 
the box plot function of the STATGRAPHICS package, the 
ie distribution appeared similar to the Normal 
distribution (Figure 11.) The Normal hypothesis was tested 
Mme the Chi-Square Goodness-of-Fit test and the 
Kolmogorov-Smirnov (K-S) test statistic, both found within 
di WSTATGRAPHICS library. A histogram of the data was 
plotted and appears in Figure 12. 

Mmeem@hi=Sclare test proved to be significant at the 
ST level and the K S test had an approximate significance 


level of 0.99999. Fam "rm is the complete results of the 
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Figure 12. Histogram Of Primary Flight Grades With 
Normal Distribution Curve Overlay 


eo GAL SQUARE TEST RESULTS OF PRIMARY FLIGHT GRADES 


iChisquare Test 


Lower Upper Observed Expected 
Limit Limit Frequency Frequency Chi square 
at cr below ri e S . 090331 
3.82 3.04 8 8 . 00087 1 
2.04 5. 065 11 12 2022213 
3.06 3.08 10 11 . 088258 
above SAAB DE li 969912 


Chisquare = 0.271585 with Z d.f. Siq. level = 8.875024 


Wioi-Square test and Table VII is the outcome of the K-S 
test. Both test indicate that the primary flight grades are 
feestributed approximately according to a Normal 
ms trl1but2on. However, the data may well be described by 
ether distributional families. 

Initial analysis of the selected 55 observations 
revealed two passing grades which were below the 3.0 passing 
meant ograde minimun, observation numbers 6 and 27. Contact 
ME both NAMRL and the appropriate flight training 
sguadrons did Psi resolve this inconsistency. Therefore, 
these points were considered entered in error and deleted 
from the study. 

Since only those subjects who pass are given a 
WéWéWéht grade there is an distinct truncation of grades at 
the 3.0 level. Table VIII provides the sample statistics for 


this variable. 
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Estimated KOLMOGOROV statistic 
Estimated KOLMOGOROV statistic 
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overall statistics 
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Approximate significance level 
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TE MMA KEGEL S TON ANALYSIS AND RESULTS 


This chapter discusses the regression model used for the 
final stage of the analysis and the method used to deal with 


missing values in the dependent variable. 


Ar MULTIPLE LINEAR REGRESSION MODEL 
The linear regression model used for this study was of 
the form: 


Y = PE 


where, Y is a vector of response (dependent) variables 
B is a vector of coefficients or weights 


X is a matrix of explanatory (independent) 
variables 


€ is an error term 


In using this model certain assumptions were made 
concerning the error term [Ref. 8]: .LS1 


1) they were assumed to be independent random 
variables; 


2) they were assumed to have mean zero with constant 
variance o2; and, 


3) they were assumed to be Normally distributed. 
Unfortunately, not all of these assumptions can be easily 
checked. 

Avoiding an over-determined model was the first concern 


in the regression analysis. To reduce the large number of 
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variables available, the step-wise regression routine was 
used. This routine selected the "most significant 
variables based on the impact each had on the model. Im 
this case 11 of the variables (VAR9, VAR2I, VAR24, VARCO 
VAR26, VAR27, VAR33, VAR34, VAR36, VAR39, AND VAR40) were 
designated as the most significant. Since the U. S. Navy is 
not likely to delete the AQT test from its inventory, it was 
included with little impact on the overall effectiveness of 
the model. 

The results of the initial regression are displayed in 
Table IX. This regression omitted any observations ties 
were missing the primary flight grade. This effectively 
eliminated any failed scores from entering the regression, 
which can introduce biases. The overall R-Square was 
0.618882. When adjusted for the number of degrees of 
freedom the R-Square, commonly referred to as the adjusted 
R=Sġuüare,- dropped to 0. 4o44, The adjusted R-square miis 
detained as (Ret. 9:pp. 17-1727: 

(e'e nu 

R-Square (adj.) = 1 - ——————————————— 
(y-vbar) (7 -ybar Mc 
where, y is a vector of the observed dependent variables 


ybar is the scalar representing the mean of the 
observed y 


e is the residual of the observed values of y minus 
the predicted values values of y 
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NS e number observations 
RTS "the number "or varlables entered into the model 


symbolizes matrix transpose. 


MM EUE' TY REGRESOTON WITH CENSORED FLIGHT GRADES 





Independent variable coefficient std. error t-value sia.level 
Constant 2.8892 7% 0.050654 zer deln 0.0000 
VARS -0.001384 0.002387 nde 3 0.6462 
VAR9 0.00717 0.0025 MSS E 0.0070 
VAR21 0.00105 0.00037z Lc Ea 2.020773 
VAR24 0002358 0.016273 =1. 7336 0.0921 
VAR25 G.001106 0.000564 In o. 0. 0380 
VAR26 GT Oo 0. 008038 1.9684 0. 1:60 
VAR27 ^ 207007 156G 0.010119 -0.7074 0.409041 
VAR33 0013333 0.008828 Tm a 0.12436 
VÀR34 -@. 0035264 0.008585 210731 ub: 
VAR36 0.0010z8 0. 00033 5.013: Qo. 0018 
- VAR39 154 °788-606 91826 3E-6 202 70:31 0. 4863 
‘ VAR40 -6.699135E-6 2, 020554E-6 == lees Oi Wie ae 
ne eC 
R-S0. (ADJ.) = 0.4844 Gs 0.022341 MAE= 0.014695 DurbWat= 2.1139 
Frevimusly: 0.0000 0.000000 o.000000 0.000 
“7 observations fitted, forecasts) computed for 6 missing val. of dep. var. 
Meme roess-valrdation, the coefficients of this 


regression were used with the variables from the remaining 
56 observations. The results of the plot of the observed 
primary flight grades versus the predicted primary flight 
grades can be seen in Figure 13. While this does not 
Wiéeddcate “perfect” predictability, there is still an evident 


relationship. 


21 


[c 4 
200 SERRE a RAR TODO TEE = = e — E EE 
F Be E joc a 
y» 3.052 EEEE vu EE E = 
E = 3 . e SINE >^ = 
c aE ————— — NS — MA ein t 2 
- a. = 
i Y 
A TE Sr EBENEN e al E tenen 3 
2 9 EL 20 0 be. 2 1 O YO AO A 
aa 2.9) 3 3303 Jot goo 

YHRT 


Figure 13.1 YHA VS — OE 


In an effort to show how representative the predicted y 
values were, the correlation coefficient was calculs 
using [Ret..102p.0213]1 

Covariance(yobs, yhat) 


r{yobs,yhat) = 
c(yobs) c(yhat) 


For this regression the correlation was .4216; 


indicating moderate correlation between the predicted values 


and the observed. 
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For comparison a regression was performed using oniy the 


AQT (VAR8) and FAR (VAR9) tests scores, which are the only 


tests currently administered for aviation selection. 


This regression, appearing in Table X, shows an R-Square 


peor. 764 and an adjusted R-Square of only 0.1389, 


considerably lower than the previous regression. 


TABCETX. REGRESSION OF CURRENT TEST 


Independent variable 





coefficient std. error t-value s1Q.level 
“e aa a 0_04_1.....:;-_—_ li 
Constant E 3.026862 0.023888 126, 7035 O. 0006 
VAR8 =O), 005617 0.00312 Ia 0.25526 
VARS @.@07795 o. 007333 3.0037 0.0044 
Raf, (ADJ.) = 0.1383 SE= 0.028870 MAE= 0.022633 DurbWat= 1.561 
Previcusly: 0.0009 0. 000000 0.000000 0.000 
47 observations fitted, forecast(s) 


computed for 6 missing val. of dep. var. 


Gross - Validation fared no better with the coefficients 


from this limited testing. Figure 14 displays a total lack 


Semepredictive quality in this model. The correlation 


Metficient for this model was 0.0762, indicating little, if 


any, correlation between the predicted values and the 


observed values. 
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Figure 14. YHAT vs. ES 


Be 115 NGO GOES 

Of the 53 observations, six involved subjects whoMdea 
not complete flight training, and, therefore, did not ENE 
associated primary flight grades. These missing values, 
thus deleted observations, could hold valuable inpuu 
model and warranted further study. 

The first attempt to estimate these missina values 
relied on the primary flight grade distribution's sima4eau E 
to the Normal distribution. If, indeed, the failures could 
be attributed to low flying grades (those below 3.0), then 
using the sample's mean and variance an expected value for 
the Normal tail area below the 3.0 level could be computed. 

According to current studies dealing with this phenomena 


(e.g., Little and Rubin, 1983; Wachter and Trussel,) TOE 
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using the mean and variance from the truncated sample may 
bias the estimation of the the missing values. Little and 


Rubin recommend using [Ref. ll:pp. 218-220]: 


“~~ = 
Falsa bar 
02 = S2(n/N) 
where, n represents the number of uncensored observations 


N is the total number of observations 

eee is the sample average 

S2 is the sample variance 
EHE assumption for this estimation is that the values are 
missing at random. 

Je a S randomness restriction and the data s 
Aros imationm to the Normal distribution in mind, the 
missing data were evaluated along the same lines as 
recommended above. Since all the data were missing from the 
tail, i.e., grades below 3.0, the conditional expected value 
the nissing points could be estimated by taking the 
expected value of any point in the area below the 3.0 cut- 
off, given that the point actually appeared below that cut- 


off, or, mathematically: 
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DONE UAM e ^4 CO7u)/o0) ? ax 


> 


miss 


1/Y2n0 ¿7% ((X-u)/o) ? ax 


where, Gr is the minimum grade required to pass primary 
Plight tscnools 

This yielded a value of 2.989 to be used in place of the 
missing values during the regression analysis. The results 
of this regression appear in Table XI. 

Notice the smaller R-squared term, 0.3988, in this model 
as compared to the previous model when the missing values 
were simply omitted. 

Of note, however, is that during the cross validation 
procedures the correlatior between the observed responses, 
YOBS, and the predicted responses, YHAT, was higher using 
these values than any other values. Unfortunately, the 
value, 0.4246, still represented a relatively low number. 


Figure 15 shows the linear relationship between the 


predicted values and the observed values. 
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Independent variable coefficient “sve. error t-value 
Constant 2.875482 Q. 044126 65.1654 
VAR8 -0. 0007253 0.0025 -D. 23956 
VARI 0.006458 Q. 802264 2.8520 
VAR21 0.001102 D. 000533 645113 
VAR24 -0.0253235 0.014835 -1.7451 
VAR25 0.001438 0.000:08 6.9016 
VAR26 @.01397 0.006142 zs 
VAR27 -0. 01055 0. 00757 -1.3346 
VAR33 0.015411 0.007703 2.0005 
VAR34 -0.011056 D. 007237 TS 
VAR36 0.001034 9.000317 Se 
VAR39 -5.692148E-6 5.636932E-6 -1.0028 
VAR42 -6.558864E-6 2.630838E-6 -2.4331 

E-50. CADJ.) = @.6343  SE- 0.021067 MAE= 0.014470  DurbWat- 

Frevicuslyv: 0.48-14 Q. 02722341 0.014635 
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This method relies heavily on the assumption that all of 
the missing values lie below the minimum passing flight 
grade of 3.0. This assumption is weakened by the fact that 
poor flight performance is not the only motivating factor HN 
dropping a student from the flight program. In fact In mon 
cases those students who are dismissed from flight training 
are performing well in the cockpit of an aircraft, but lack 
the motivation or aggressiveness needed to be a Naval pilot 
or simply do not meet the academic standards required in 
Naval aviation. In these cases it is reasonable to assume 
that the test results for such an individual could indic 
a propensity to do well in flight training, but nonetheless 
be listed as a failure in the data set. This indicated that 
another method for replacing missing values had to be 
evaluated. 

This second, and last, method used for estimating the 
value of the missing flight grades is taken from Yates. 
During this procedure coefficients of all available 
variables were calculated using the multiple regression 
£ormnula (Fet l2:Dpï Ol SO]: 

Y = B* Made 


where, Y is a vector of the flight grades with missing 
values not represented by estimation 


B* is a vector of coefficients or weights 
X is a matrix of explanatory variables 


t is an error term. 
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These B* coefficients are then used to predict the value 
of those data points that are missing. The new, and 
complete, set of flight grades, Y*, are then entered into a 
separate regression resulting in a new vector of f weights. 

Yx = B X+ e 
where, Y* is the vector of flight grades which contains the 
estimated missing values. All other symbology remains 
unchanged in translation. 

What made these missing value estimations unique was 
that unlike the other method for estimating these values, 
they were not below the failing grade of 3.0. This method 
might help compensate for those individuals who either 
voluntarily withdrew or were involuntarily withdrawn for 
ether than poor flight performance. Reasons for the "fail" 
indication did not accompany the data, making verification 
of these high-mark missing values impossible. 

Using this method, Table XII, the R-square reached its 
meomest value, 0.76531, with an adjusted R-square of 
0.69490. With nearly 77% of the variance in the model 
explained, these coefficients were used in cross validation 
with surprising results. 

Although the plot of the data, Figure 16, appears to 
show some strong predictability, the correlation of the YHAT 


and YOBS was lower, 0.3910, than that of either the model 
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without estimation or the model using the Normal 


approxımatıoar 


TABLE XII. REGRESSION USING MISSING VALUE ESTIMATES 
CALCULATED USING THE YATES APPROACH 





Independent variable coefficient std. error t-value sia.level 
Constant 2.93446 0.067756 43.2033 o. 02000 
VAR8 0.003707 0.003835 0.365 0.3400 
VARI 0.005548 0.003477 1.535 2.1184 
VAR21 2.020788 0.000511 1.5407 0.192179 
VAR24 -D.0488 0.022872 -2.133 0.0321 
VAR25 -0.0003ž8 0.00032 -1.0244 0.3118 
VAR26 0.011323 0.203431 1.2006 0.2270 
VAR27 -0. 20434 2.011624 -0.4250 0.6731 
VAR33 -2. 000258 0.011823 -0.0218 0.2827 
VAR34 @.011545 0011204 1.030: 0. 3030 
VAR36 0.0007 0.000486 1.4404 0. 15/208] 
VAR39 -7.228588E-6 8.655863E-6 -0.8351 0.4086 
VAR4@ -5.496087E-6 4.03368E-6 -1.3605 0.1813 

E-SD. (ADJ.) = 0.2185 SE= 0.022243 MAE= 0.021708 DurbWat= 1.752 

Previeusly: o. 0000 0.900000 0. 000000 0.000 


oo abservaticons fitted, forecast(s) camputed for Ø missing val. af dep. var. 


Flot of YOBS vs YHST WITH MISSING VALUES 
REPLACED BY YATES APPROACH 
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Figure 16. Yhat vs. Yobs 
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SY POTHESTS OF CORRELATIONS 

The correlation coefficients of the three methods were 
subjected to a test to determine if they were from the same 
population. 


H rho] = rho» - rho4 


Gi 
Hy: Not un 

ane initial calculation, taken from Snedecor and 

@eenran, involves the transformation of the correlation 


coefficlent, r, to a quantity designated z, given by [Ref. 


Dp. 185-187]: 
z*— Uum (2 tor) = in (Lx)! 


Dentes estatistic, U, is distributed as a Chi-Square 


with (k-1) degrees of freedom, where k is the number of 


merrelations to be tested. The statistic is given by: 
k k K 
Do E Wi zx cet ( 2 cee) E 
qz] gd 1-1 
where, wi is no o 


KE is the number of observations 
k is the number of correlations to be tested 


z is the z value given by the equation listed above. 


The computations for this study appear in Table XIII. 


As presented, there is not sufficient evidence to reject the 


al 


null hypothesis that the correlations are drawn e 


common population correlaciones 


TABLE XIII. TEST OF HYPOTHESIS OF COMMON POPULAR Tee 


CORRELATION 

Sample “mM n-—3 r | Z l (n-3)z | nz 
Censored | 53 ı 50 i .4216 i .4496 ı 22.4800) 107782 
Normal i i i i i i 
Approx MODEM 52 | .4246 | .4533 | 23.5716; 109565 
Yates Bussi, "52 1 .3910 1. .4115 | 21.3960! 8 S880 

Total 631 3549 | | 67.44761 29.5578 
x (Ealenlarea) = 0.0572 
221. 952. 4 1220). -202 12026 


Table XIV is a summary table showing the coefficients 
and related R-Square/adjusted R-Square for each of the 
methods used in this analysis. In any case there is ample 
evidence that methods in dealing with missing values should 
be investigated ary time studies of this type eam 


undertaken. 
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SUMMARY OF 
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REGRESSIONS 


NORMAL 
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TABLE XIV: 
CURRENT! 
Coefficients 
Constant 3.0268 | D 
VAR8 -3.62E-3! -1. 
VARSI OEN Ue 
VAR21 1 
VAR24 -0 
VAR25 al 
VAR26 0 
VAR 27 -7 
VAR33 O 
VAR34 -9 
VAR36 d 
VAR39 -4 
VARAO -6 
R- SQUARE 02176385, O. 
R-SOUARE 0.138e90! 0. 
(ADJ) 
CORR 02076207 On 
(HAT, YOBS) 
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V. —CONGEUSTOHNS 


Psychomotor testing is not new, and had not fallen 
disfavor for lack of predictive validity when it was dropped 
from the selection process. The decision to eliminate this 
test was based mostly on its size and the difflc EEEg 
encountered in calibrating the equipment. Such is noto" 
case with today's computer-assisted tests. The equipment is 
easily transported ana maintained. 

Tn this study. certain aspects of the psychomotor test 
as a predictor became clear. First, the replacement of 
missing dependent values, at least in this study, had little 
impact on the overall success of the model, if success is 
measured by the amount of correlation alone. If, however, 
the measure of success is the amount of variation explained 
by the model, then the impact was sign ee Seconds 
specific elements of each of the two series of tests made 
statistically significant contributions to the selena 
Criterion, Primary Flight Gradecr Third, The Academia 
Qualification Test did little to predict the scores Cie. 
student aviators in primary flight “r'r Un Fourth, ane 
most importantly, psychomotor testing, in gcn T eE 
dramatically increased the predictability of primary fI 
grades over the limited system of pen-and-paper tests 


currently in use by the U. S. Navy. 
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The replacement of missing data points presented an 
interesting paradox. In this sample the replacement had no 
mcan beari cn “tne predictability of the model, but 
did indicate a dramatic increase in the R-Square term. 

While the Snedecor/Cochran method showed that the 
correlation coefficient of each model was from the same 
population correlation, there were considerable differences 
in the R-Square term associated with each approach. For 
Mm tance, t£ it is assumed, as indicated, that the 
correlations are from the same population, then using the 
model which explains the most variance would be the logical 
choice (the Yates method). If, however, these correlations 
are not assumed from the same population then high R-Square 
of the yates approach and the high correlation coefficient 
of the Normal approximation method may be overshadowed by 
the consistency displayed in the model using the censored 
delight grades. Eres model theWR-Sguare is still 
mele ve lv high, 0.62, and the correlation coefficient, 
INS) 18 only slightly lower than the highest, 0.4246. 
These results may well be indicative solely of this data and 
bears close scrutiny in future studies in this arena. 

Most of the variables chosen through the stepwise 
regression were intuitively appealing. It seems rational to 
believe that one's ability to perform well in the air can be 


linked to one's ability to perform well on a tracking task 
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when one's attention is focused elsewhere (Multi-dimensional 
dual task Compensatory Tracking.) Others, however, were not 
as intuitive, e.g., the standard deviation of the time 
answer the absolute difference test numbers correctly. 

Notice that the variables included in the regression 
models were divided between the two series of tests, 
Gibb/Damos and Griffin/Mosko. While it appears that the 
Gibb series contributed the greatest number, it is worth 
remembering that only 3 of the 7 variables are Ma 
different tests, e.g., the VAR24, VAR25, VAR26 and VAR2Z/ ame 
part of the Single Task Absolute Difference test. Tt 
difficult to ascertain the effect of deleting any one 
portion of either of these series of tests, even though the 
variable does not enter the model. It is recommended» 
therefore, that both series be administered in their 
entirety. 

Of equal interest is the exclusion of the AQT scores. 
In each of the models the AQT had the lowest significance 
level. In fact, when it was removed from the model the 
correlation coefficients of the three models did not change 
significantly. This would seem to indicate that the AQT does 
not do well in predicting flight cc This in no way 
reflects unfavorably on the test, which may do wel lia 


predicting a strict pass/fail in the flight training oe 
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“a nal, 1h predicting an individuals acceptability as a 
Naval Officer. 

While the inclusion of these variables are as 
interesting as they are puzzling, the real interest lies in 
the model's increased predictability over the current 
procedures. As seen above, the current tests, the AQT and 
the FAR, have low correlation with the flight grades of the 
subjects. It is recognized, however, that these pen-and- 
paper tests are extremely portable and convenient for 
administering to potential aviators at college campuses, 
@areer fairs, etc. The inclusion of computer-generated 
tests may cause logistical problems in the transportation of 
the testing equipment. 

Mate difficulty can be circumvented easily by 
administering the pen-and-paper tests at remote sites. This 
may require lowering the minimum AQT/FAR scores to avoid 
wering out those individuals who would have otherwise 
done well on the psychomotor tests and, presumably, in 
echt training. Those individuals who meet these revised 
standards for aviation would then be eligible to take the 
computer-assisted test administered at the Naval Recruiting 
Peestrict. 

Psychomotor tests show great potential in predicting the 
Bree of success in the flight syllabus. It is in this 


later role that these tests may eventually prove their 
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worth. Such early selection would help eliminate manpower 


shortages in the various pipelines by matching the aviator's 


ability to the type of airc am "A 200 knot mind does not 
belong in a Mach 2 aircraft.", is a quote (Fr 
unidentified aviation instructor. What is missing, however, 


is that that "200 knot mind" may do extremely well at the 
controls of a rotary wing aircraft. Psychomotor testimo SEE 


help to predict this man/machine match. 
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VE heECOMavwn LON FORFFURTHER STUDY 


Mia study barely scratches the surface of the pilot 
selection issue. The data for these tests are still being 
collected and subjects who took these tests originally are 
still being tracked. 

An excellent follow on study to this would be examining 
the pass/fail variable using the logistic regression model 


Ber:ned as [Ref. 14:pp. 1110-1117]: 


c ePX 


Pr (T=t) 


{1 + e So? 


Ima 


1=1 


where, c is the number of distinct binary vectors (in this 
case c = 1) 


X 1S a matrix of variables 
B is a vector of beta weights obtained through 
maximum likelihood estimation 
This model would produce a probability of pass/fail for 
each of the subjects, and could be used as a strong 
supplement to the current FAR/AQT testing. 
These tests offer a myriad opportunities for further 
Studies. The information available from the Naval Aerospace 
Medical Research Laboratory is theoretically limitless. 


This data alone could have been studied in numerous 
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different ways, e.g., the effect of implementing the 
throttle portion of the tracking test and era Marmo Eas 
impact on the predictive model. Decision analysis techniques 
could be employed to determine the costs of selecting a 
failure. 

Additional studies should be conducted as each of the 
subjects in this study complete additional flight train 
Of importance will be the failure rate of the tested 
subjects at advanced flight schools, e.g., advanced jet 
aırerafr trasming: 

This will add validity to the utilization of psychonmcu 


testing as a tool for pipeline selection. 
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APPENDIX C 


PLOTS OF VARIABLES 
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APPENDIX D 


SAMPLE AQT/FAR QUESTION 


ACADEMIC QUALIFICATIONS TEST (AQT) 


Verbal Analogy: 
ZINC + SULEURIG ACID 


atom : molecule 
copper : brass 


element : compound (correct ans) 
metal : salt 


molecule : mixture 


mon w » 


MECHANICAL COMPREHENSION TEST (MCT) 


Mechanical Principles: 


What is the ideal mechanical advantage of an incline that 
rises 3 feet for each 12 feet of length? 


(correct ans) 


COD 


E 
I: 
ES 
) Not enough information to answer question 


BIOGRAPHICAL INVENTORY (BI) 


Personal Interest: 


1. Which of the following interest you least: 


Football 
Water Sports 
Badminton 
Tennis 


OAD» 


do 


2. Do you enJoy team sports more than 
one) sports? 


(A) Yes 
(B) No 


mation Information: 


Lowering the flaps 


An increases wing lift (Correct Ans) 
Sey turns the aircraft 


individual (one-on- 


fete rPrepares the aircraft for fast flight 


(D) decreases wing surface area. 


SPATIAL APPERCEPTION 


we iew A indicates that the aircraft is higher than in view 


B, mark A, otherwise mark B. 
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